Skip to content

branch-3.0: [enhancement](filecache) fix default capacity and add reset_capacity validation #51711 #52152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jun 25, 2025
Merged

branch-3.0: [enhancement](filecache) fix default capacity and add reset_capacity validation #51711 #52152

dataroaring merged 1 commit into from
Jun 25, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jun 23, 2025

Cherry-picked from #51711

@freemandealer
[enhancement](filecache) fix default capacity and add reset_capacity …
fd3bc3d 
…validation (#51711)

originall default capacity is total disk size * ratio we user set zero
or extrem large space than acutal disk. I found the ratio part make no
sense. we just use total disk size in this case.

I alse add validation for reset_capacity http action rather than set the
raw capacity.

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@github-actions github-actions bot requested a review from dataroaring as a code owner June 23, 2025 13:27
@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 23, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jun 23, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 23, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jun 23, 2025

TPC-H: Total hot run time: 39511 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fd3bc3dd4bc399e09583046cb9093f7e48fcc848, data reload: false

------ Round 1 ----------------------------------
q1	17596	6777	6595	6595
q2	2047	160	175	160
q3	10600	1064	1119	1064
q4	10564	715	711	711
q5	7738	2871	2732	2732
q6	209	135	134	134
q7	949	620	597	597
q8	9350	1961	1987	1961
q9	6539	6382	6428	6382
q10	7060	2280	2276	2276
q11	469	251	270	251
q12	391	214	212	212
q13	17781	2997	3000	2997
q14	232	205	205	205
q15	523	459	477	459
q16	473	373	382	373
q17	969	559	597	559
q18	7342	6711	6564	6564
q19	1412	1088	1087	1087
q20	468	201	195	195
q21	4038	3134	3014	3014
q22	1071	993	983	983
Total cold run time: 107821 ms
Total hot run time: 39511 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6537	6543	6518	6518
q2	328	241	224	224
q3	2906	2753	2745	2745
q4	2060	1783	1793	1783
q5	5713	5729	5743	5729
q6	204	126	129	126
q7	2202	1797	1780	1780
q8	3365	3483	3535	3483
q9	8865	8744	8866	8744
q10	3566	3550	3518	3518
q11	607	489	502	489
q12	801	603	617	603
q13	10174	3107	3146	3107
q14	310	295	264	264
q15	510	468	462	462
q16	474	422	458	422
q17	1824	1640	1609	1609
q18	8189	7799	7628	7628
q19	1692	1490	1429	1429
q20	2164	1867	1808	1808
q21	5063	5083	5098	5083
q22	1134	1021	1065	1021
Total cold run time: 68688 ms
Total hot run time: 58575 ms

@doris-robot
Copy link

doris-robot commented Jun 23, 2025

TPC-DS: Total hot run time: 197348 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fd3bc3dd4bc399e09583046cb9093f7e48fcc848, data reload: false

query1	1271	912	883	883
query2	6270	1973	2002	1973
query3	10798	4428	4331	4331
query4	61279	29540	23391	23391
query5	5218	473	442	442
query6	403	183	172	172
query7	5463	315	329	315
query8	310	226	227	226
query9	8460	2577	2565	2565
query10	467	269	258	258
query11	18035	15418	15815	15418
query12	154	99	100	99
query13	1399	438	451	438
query14	10842	7668	7132	7132
query15	197	186	183	183
query16	7315	517	406	406
query17	1168	594	584	584
query18	1909	312	317	312
query19	230	165	157	157
query20	119	115	114	114
query21	205	105	105	105
query22	5126	5075	4884	4884
query23	34707	33888	34240	33888
query24	6153	2917	2941	2917
query25	542	423	442	423
query26	660	172	173	172
query27	1875	359	360	359
query28	4431	2129	2120	2120
query29	696	461	453	453
query30	229	154	160	154
query31	1024	821	825	821
query32	73	54	57	54
query33	457	304	300	300
query34	969	510	516	510
query35	830	734	707	707
query36	1087	970	937	937
query37	107	66	67	66
query38	4094	3958	3957	3957
query39	1531	1483	1486	1483
query40	204	99	95	95
query41	48	47	48	47
query42	119	101	110	101
query43	539	504	504	504
query44	1204	830	820	820
query45	183	171	167	167
query46	1167	745	728	728
query47	2045	1942	1950	1942
query48	480	386	384	384
query49	749	394	388	388
query50	832	435	431	431
query51	7410	7271	7210	7210
query52	103	95	93	93
query53	279	190	188	188
query54	585	469	462	462
query55	76	75	78	75
query56	300	271	254	254
query57	1333	1220	1204	1204
query58	235	219	231	219
query59	3346	3001	3236	3001
query60	294	272	273	272
query61	114	113	174	113
query62	784	693	697	693
query63	226	192	190	190
query64	1693	703	631	631
query65	3270	3182	3205	3182
query66	723	298	297	297
query67	15879	15640	15775	15640
query68	4396	584	573	573
query69	427	262	260	260
query70	1160	1113	1107	1107
query71	365	261	260	260
query72	6366	4035	4080	4035
query73	761	349	358	349
query74	10247	8949	9167	8949
query75	3374	2673	2662	2662
query76	2012	1073	1133	1073
query77	486	271	270	270
query78	10600	9654	9443	9443
query79	1996	604	615	604
query80	1343	423	442	423
query81	510	220	213	213
query82	1227	86	86	86
query83	274	145	150	145
query84	284	78	74	74
query85	1042	303	296	296
query86	382	296	287	287
query87	4381	4316	4173	4173
query88	3584	2386	2374	2374
query89	426	294	294	294
query90	1974	182	181	181
query91	198	149	148	148
query92	63	49	52	49
query93	2433	567	552	552
query94	776	293	309	293
query95	367	261	261	261
query96	636	290	284	284
query97	3305	3098	3123	3098
query98	214	203	194	194
query99	1567	1298	1326	1298
Total cold run time: 317499 ms
Total hot run time: 197348 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 23, 2025

BE UT Coverage Report

Increment line coverage 81.25% (39/48) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.18% (10914/26503)
Line Coverage 31.99% (93445/292109)
Region Coverage 31.05% (48158/155074)
Branch Coverage 27.52% (24679/89666)

@doris-robot
Copy link

doris-robot commented Jun 23, 2025

ClickBench: Total hot run time: 29.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fd3bc3dd4bc399e09583046cb9093f7e48fcc848, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.02	0.03
query3	0.24	0.06	0.06
query4	1.63	0.10	0.11
query5	0.54	0.51	0.50
query6	1.15	0.73	0.74
query7	0.02	0.02	0.02
query8	0.03	0.03	0.03
query9	0.57	0.51	0.50
query10	0.56	0.55	0.58
query11	0.13	0.09	0.13
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	0.79	0.78	0.81
query15	0.84	0.83	0.82
query16	0.37	0.40	0.38
query17	1.00	1.00	1.07
query18	0.24	0.22	0.21
query19	1.96	1.78	1.88
query20	0.01	0.01	0.01
query21	15.38	0.58	0.59
query22	2.15	2.17	1.78
query23	17.05	1.00	0.86
query24	3.34	1.21	0.47
query25	0.26	0.20	0.06
query26	0.41	0.13	0.14
query27	0.05	0.05	0.04
query28	10.62	0.52	0.45
query29	12.63	3.17	3.17
query30	0.24	0.07	0.07
query31	2.86	0.38	0.38
query32	3.25	0.47	0.45
query33	2.97	3.01	3.01
query34	17.14	4.42	4.52
query35	4.49	4.48	4.54
query36	0.67	0.50	0.46
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.17	0.12	0.13
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.03	0.02	0.03
Total cold run time: 104.92 s
Total hot run time: 29.41 s

dataroaring
dataroaring approved these changes Jun 25, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 2bc0c0e into branch-3.0 Jun 25, 2025
23 of 25 checks passed
@github-actions github-actions bot deleted the auto-pick-51711-branch-3.0 branch June 25, 2025 01:43
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @dataroaring @freemandealer